AirLLM: Run a 70B LLM on a Single 4GB GPU (How It Works, and the Catch)
AirLLM lets a 4GB GPU run a 70B parameter model by streaming layers from disk instead of loading them into VRAM. Here's how it works, real model-to-VRAM numbers, and the speed tradeoff nobody mentions in the headline.